Skip to content

feat(cli,format): deterministic JSON envelope with plumb_version, run_id, summary - #123

Merged
aram-devdocs merged 1 commit into
mainfrom
claude/issue-31-json-deterministic
Apr 25, 2026
Merged

feat(cli,format): deterministic JSON envelope with plumb_version, run_id, summary#123
aram-devdocs merged 1 commit into
mainfrom
claude/issue-31-json-deterministic

Conversation

@aram-devdocs

Copy link
Copy Markdown
Owner

Summary

Closes #31. Wraps plumb lint --format json output in a deterministic envelope:

{
  "plumb_version": "0.0.1",
  "run_id": "sha256:<hex>",
  "summary": { "error": N, "info": N, "total": N, "warning": N },
  "violations": [ ... ]
}
  • plumb_version — compile-time env!("CARGO_PKG_VERSION") of plumb-format.
  • run_idsha256:<hex> of the compact serialization of the sorted violations array. Whitespace and version bumps don't shift it; any observable violation change flips it.
  • summary — counts by severity, alphabetically keyed.
  • violations — defensively re-sorted by Violation::sort_key() before hashing and emit.

json() builds the envelope with explicit alphabetical inserts so output stays stable regardless of whether serde_json/preserve_order is toggled in the workspace (currently enabled transitively via schemars).

Acceptance criteria

  • 3x byte-diff deterministic — covered by just determinism-check and a new tests/determinism.rs suite (per-formatter byte-identity tests for json/pretty/sarif/mcp_compact).
  • run_id derivation documented — rustdoc on json() describes the hash input precisely.
  • JSON validates against plumb-output.schema.json — the schema file does not yet exist in-tree; this PR ships the envelope shape such a schema would describe. Flagged for a follow-up issue.

Test plan

  • cargo nextest run -p plumb-format
  • cargo nextest run -p plumb-cli (existing CLI integration JSON test still finds "rule_id" inside violations[])
  • cargo clippy --workspace --all-targets -- -Dwarnings
  • cargo fmt --check
  • just determinism-check
  • just validate end-to-end

🤖 Generated with Claude Code

…_id, summary

Wraps the JSON formatter output in an envelope object with
`plumb_version`, a content-derived `run_id`, a `summary` stats block,
and the sorted `violations` array. The `run_id` is `sha256:<hex>` of
the compact serialization of the sorted violations, so whitespace
tweaks and version bumps never shift the digest while any observable
change in a violation flips it.

Adds a `tests/determinism.rs` suite that locks in byte-identical
output across runs for every formatter, covers the envelope shape,
and asserts that re-ordering inputs leaves `run_id` unchanged.

Closes #31.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Now I have everything I need for a complete review. Here is my full assessment:


Review: feat(cli,format): deterministic JSON envelope with plumb_version, run_id, summary

Crates touched: plumb-format only.


1. Determinism Invariants

PASS. No wall-clock sources, no HashMap/HashSet in output paths. serde_json::Map with explicit alphabetical insertion is the right defensive move given the transitive serde_json/preserve_order activation via schemars. The run_id derivation is correct: compact-serialize the sorted violations array (not the pretty envelope), then SHA-256 hex-encode — so whitespace tweaks and version bumps never shift the digest. Defensive re-sort before hashing is correct.

sha2 = "0.10" is declared in [workspace.dependencies] and is MIT-licensed (in the deny.toml allowlist). Layer is clean: plumb-formatplumb-core + external only.


2. Workspace Layering

PASS. #![forbid(unsafe_code)] at src/lib.rs:18. No internal cross-crate direction violations.


3. Error Handling

PASS. All serde_json calls use ?. No unwrap/expect in library code (#![deny(clippy::unwrap_used, clippy::expect_used)] at src/lib.rs:20 enforces this).


4. Test Coverage — Issues

Warning 1 — dead test; the reordering invariant is never actually exercised.

crates/plumb-format/tests/determinism.rs:89:

if violations.len() < 2 {
    return; // canned fixture too small to reorder
}

The golden snapshot at format_snapshots__json.snap:11 confirms the canned fixture produces exactly 1 violation ("total": 1). So violations.len() < 2 is always true and the reordering body never runs. The test json_run_id_is_stable_under_input_reordering is permanently silent — it never asserts anything.

Fix options (pick one):

  • Replace the fixture() call in this specific test with a hand-built Vec<Violation> of length ≥ 2, as is done in other determinism tests.
  • Change return to panic!("canned fixture must produce ≥ 2 violations for this test to be meaningful") so a fixture regression is loud.

Warning 2 — plumb_version is a volatile field baked into a golden snapshot.

crates/plumb-format/tests/snapshots/format_snapshots__json.snap:6:

"plumb_version": "0.0.1",

Every routine version bump (0.0.1 → 0.0.2) will fail just test with an insta diff that has zero content significance. The testing rule (.agents/rules/testing.md) explicitly covers this: "insta::Settings can redact volatile fields — use it for paths and timings when you can't avoid them." plumb_version is in the same category as a path or timestamp.

Fix: in format_snapshots.rs, wrap the json_snapshot test with:

let mut settings = insta::Settings::clone_current();
settings.add_redaction(".plumb_version", "[version]");
settings.bind(|| insta::assert_snapshot!("json", out));

The json_envelope_has_required_fields test in determinism.rs already checks !plumb_version.is_empty() structurally, which is the right place for that invariant. No value in also locking it in a snap file.


5. Documentation

PASS. json() has thorough doc including # Envelope, # run_id derivation, and # Errors. PLUMB_VERSION has an explanatory comment. hex_digest (private) is slightly confusingly named — it takes bytes-to-hash, not a pre-computed digest — but it's private and the comment clarifies it. Minor nit, not actionable.


Punch List

# File:Line Severity Issue
1 tests/determinism.rs:84-99 Warning json_run_id_is_stable_under_input_reordering silently no-ops forever because fixture() yields 1 violation; the reordering invariant is never asserted
2 tests/snapshots/format_snapshots__json.snap:6 Warning plumb_version is a volatile field locked in a golden snap; will fail CI on every routine version bump

Verdict: REQUEST_CHANGES

@aram-devdocs
aram-devdocs merged commit 8f2d9c3 into main Apr 25, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(cli,format): --format json deterministic key ordering

1 participant